Systems and methods for targeted advertisement insertion into a program content stream

ABSTRACT

A computerized system for targeted ad insertion receives a program content stream and detects images in the content stream. The system recognizes at least one visual object within the images and matches the recognized visual object to an advertisement that correlates with the recognized visual object. The system may then select an advertisement that advertises, correlates to, is directed to or is otherwise relevant to products or services falling under the theme, topic, category of interest and/or relevant viewer demographic and inserts an identifier of the matched advertisement at an insertion point into the program content stream. This enables playback of the matched advertisement during playback of the program content stream at a time that correlates to the insertion point into the program content stream of the identifier of the matched advertisement.

TECHNICAL FIELD

This disclosure generally relates to the technology of providingtargeted advertisements, and more particularly, to automated targetedadvertisement insertion into a program content stream.

BRIEF SUMMARY

Program distributors deliver audio and/or video program content to itssubscribers. The program content includes, for example, televisionprograms, multimedia services, and the like, and may further includeadvertisement content. The program distributors typically deliver theaudio/video content to its subscribers in an industry-standard format,for example, the Moving Picture Experts Group (MPEG) MPEG-4 transportstream format.

The Society of Cable Telecommunications Engineers (SCTE) developstechnology standards related to cable telecommunications engineering.The SCTE 35 standard defines the splicing of an MPEG-4 transport streamfor the purpose of digital program insertion. The SCTE 35 standard alsodefines the messages and structure that allow the program distributor tosplice, or insert, advertisements and other digital program content intothe audio/video content.

Subscribers often have a receiving device, such as a set-top box, whichmay include a digital video recorder (DVR) or other similar contentrecording functionality. Advertisements may have different applicabilityor relevance to the viewer based on the content of the program content.However, selection of particularly relevant advertisements based on theprogram content is often not accurate and not particularly specific toactual objects, activities or scenes in the program. Also, automaticallytying product placements in program content to particular advertisementsto be played during breaks in the program content in an efficient andaccurate manner provides an efficient way to provide targetedadvertising. Thus, the embodiments disclosed herein improve thetechnology of targeted advertising by increasing the speed, accuracy andflexibility of selection and playing of targeted advertisements.

A computer implemented method for targeted ad insertion into a programcontent stream may be summarized as including receiving, by acomputerized system for targeted ad insertion, the program contentstream; detecting, by the computerized system for targeted ad insertion,images in the content stream; recognizing, by the computerized systemfor targeted ad insertion, at least one visual object within the images;matching, by the computerized system for targeted ad insertion, therecognized at least one visual object within the images to anadvertisement that correlates with the recognized at least one visualobject; and inserting, by the computerized system for targeted adinsertion, an identifier of the matched advertisement at an insertionpoint into the program content stream, enabling playback of the matchedadvertisement during playback of the program content stream at a timethat correlates to the insertion point into the program content streamof the identifier of the matched advertisement.

The method may further include initiating, by the computerized systemfor targeted ad insertion, playback of the program content stream;recognizing, by the computerized system for targeted ad insertion, theidentifier of the matched advertisement during playback of the programcontent stream; accessing, by the computerized system for targeted adinsertion, content of the matched advertisement in response to therecognizing the identifier of the matched advertisement during playbackof the program content stream; and inserting, by the computerized systemfor targeted ad insertion, the content of the matched advertisement intothe program content stream during the playback of the program contentstream.

The method may further include playing, by the computerized system fortargeted ad insertion, the content of the matched advertisement duringthe playback of the program content stream at a time during playback ofthe program content stream that correlates to the insertion point intothe program content stream of the identifier of the matchedadvertisement. The initiating playback of the program content stream;recognizing the identifier of the matched advertisement during playbackof the program content stream; the accessing content of the matchedadvertisement in response to the recognizing the identifier of thematched advertisement during playback of the program content stream; andthe inserting the content of the matched advertisement into the programcontent stream during the playback of the program content stream may beperformed by a receiving device at a customer premises. The playing thecontent of the matched advertisement during the playback of the programcontent stream at a time during playback of the program content streamthat correlates to the insertion point into the program content streamof the identifier of the matched advertisement may be performed by areceiving device at a customer premises. The receiving the programcontent stream; the detecting images in the content stream; therecognizing at least one visual object within the images; the matchingthe recognized at least one visual object within the images to anadvertisement that correlates with the recognized at least one visualobject; and the inserting, the identifier of the matched advertisementat an insertion point into the program content stream, enabling playbackof the matched advertisement during playback of the program contentstream at a time that correlates to the insertion point into the programcontent stream of the identifier of the matched advertisement may beperformed by a content server outside a customer premises. The matchingthe recognized at least one visual object within the images to anadvertisement that correlates with the recognized at least one visualobject may include determining the recognized at least one visual objectin the images is a product advertised in the advertisement; andrecognizing there exists a correlation between the recognized at leastone visual object and the advertisement based on the determination. Thematching the recognized at least one visual object within the images toan advertisement that correlates with the recognized at least one visualobject may include determining the recognized at least one visual objectin the images represents an action sequence that relates to a product orservice advertised in the advertisement; and recognizing there exists acorrelation between the at least one visual object and the advertisementbased on the determination. The matching the recognized at least onevisual object within the images to an advertisement that correlates withthe recognized at least one visual object may include determining therecognized at least one visual object in the images is one or more wordsrepresenting a product or service advertised in the advertisement; andrecognizing there exists a correlation between the at least one visualobject and the advertisement based on the determination. The recognizingthe at least one visual object within the images may include detecting aplurality of visual object candidates within the images; determining atleast one of the visual object candidates is a type of visual objectassociated with one or more of a plurality of advertisements; andrecognizing the at least one of the visual object candidates as the atleast one visual object within the images based on the determination.The matching the recognized at least one visual object within the imagesto an advertisement that correlates with the recognized at least onevisual object may include accessing a plurality of stored advertisementdata corresponding to a plurality of different advertisements;determining which advertisement of the plurality of advertisementsadvertises a product that correlates with the recognized at least onevisual object within the images based on the accessing of the pluralityof stored advertisement data; and matching the recognized at least onevisual object within the images to the determined advertisement of theplurality of advertisements.

The method may further include detecting, by the computerized system fortargeted ad insertion, audio in the content stream; recognizing, by thecomputerized system for targeted ad insertion, at least onecharacteristic of the audio; matching, by the computerized system fortargeted ad insertion, the recognized at least one characteristic of theaudio to an advertisement that correlates with the recognized at leastone characteristic of the audio; and inserting, by the computerizedsystem for targeted ad insertion, an identifier of the matchedadvertisement at an insertion point into the program content stream,enabling playback of the matched advertisement during playback of theprogram content stream at a time that correlates to the insertion pointinto the program content stream of the identifier of the matchedadvertisement.

A system may be summarized as including at least one computer processor;and at least one memory coupled to the at least one computer processor,the at least one memory having computer-executable instructions storedthereon that, when executed by the at least one computer processor,cause the at least one computer processor to initiate playback of aprogram content stream; recognize an identifier of an advertisementduring playback of the program content stream, the identifier havingbeen inserted at an insertion point into the program content stream inresponse to at least one visual object recognized within images of thecontent stream being matched to the advertisement which correlates withthe recognized at least one visual object; access content of theadvertisement in response to the recognizing of the identifier of theadvertisement during playback of the program content stream; and insertthe content of the advertisement into the program content stream duringthe playback of the program content stream.

The computer-executable instructions, when executed by the at least onecomputer processor, may further cause the at least one computerprocessor to play the content of the advertisement during the playbackof the program content stream at a time during playback of the programcontent stream that correlates to the insertion point into the programcontent stream of the identifier of the advertisement. The accessingcontent of the advertisement in response to the recognizing theidentifier of the advertisement during playback of the program contentstream may include accessing content of the advertisement at a set-topbox from a remote advertisement server. The accessing content of theadvertisement in response to the recognizing the identifier of theadvertisement during playback of the program content stream may includeaccessing content of the advertisement by a set-top box from localstorage of the set-top box. The computer-executable instructions, whenexecuted by the at least one computer processor, may further cause theat least one computer processor to receive a plurality of storedadvertisement data corresponding to a plurality of differentadvertisements; search the plurality of stored advertisement datacorresponding to a plurality of different advertisements foradvertisement data that is identified by the identifier of theadvertisement during playback of the program content stream; and accesscontent of the advertisement in response to the recognizing theidentifier of the advertisement during playback of the program contentstream based on the advertisement data that is identified by theidentifier of the advertisement during playback of the program contentstream.

A non-transitory computer-readable storage medium havingcomputer-executable instructions stored thereon that, when executed byat least one processor, may cause the at least one processor to access aplurality of stored advertisement data corresponding to a plurality ofdifferent advertisements; determine which advertisement of the pluralityof advertisements advertises a product that correlates with at least onevisual object recognized within images of a program content stream basedon the accessing of the plurality of stored advertisement data; andmatch the at least one visual object recognized within the images to thedetermined advertisement of the plurality of advertisements.

The computer-executable instructions, when executed by the at least oneprocessor, may further cause the at least one processor to insert anidentifier of the determined advertisement of the plurality ofadvertisements at an insertion point into the program content stream,enabling playback of the determined advertisement of the plurality ofadvertisements during playback of the program content stream at a timethat correlates to the insertion point into the program content streamof the identifier of the determined advertisement of the plurality ofadvertisements. The computer-executable instructions, when executed byat least one processor, may further cause the at least one processor toaccess content of the determined advertisement of the plurality ofadvertisements in response to recognizing the identifier of thedetermined advertisement of the plurality of advertisements duringplayback of the program content stream.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1A is a block diagram illustrating an overview of a system fortargeted advertisement insertion into a program content stream,according to one embodiment.

FIG. 1B is a block diagram illustrating various example components of asystem for targeted advertisement insertion into a program contentstream, according to one embodiment.

FIG. 2A is a signal diagram illustrating a program content streamincluding an advertisement insertion cue and an avail, according to oneembodiment.

FIG. 2B is a signal diagram illustrating a program content streamincluding an advertisement insertion cue with an identifier of anadvertisement inserted into the advertisement insertion cue, accordingto one embodiment.

FIG. 3 is a signal diagram illustrating a program content streamincluding an advertisement inserted into the avail based on anidentifier of an advertisement inserted into the advertisement insertioncue, according to one embodiment.

FIG. 4 is a flowchart illustrating a method for inserting an identifierof the advertisement into the program content stream during the playbackof the program content stream, according to one embodiment.

FIG. 5 is a flowchart illustrating a method for inserting the content ofthe advertisement into the program content stream during the playback ofthe program content stream, according to one embodiment.

FIG. 6 is a flowchart illustrating a method for matching a visual objectrecognized within the program content stream to an advertisementaccording to one embodiment.

In the drawings, identical reference numbers identify identical elementsor elements in the same group and class. The sizes and relativepositions of elements in the drawings are not necessarily drawn toscale. For example, the shapes of various elements and angles are notnecessarily drawn to scale, and some of these elements are enlarged andpositioned to improve drawing legibility. Further, the particular shapesof the elements as drawn are not intended to convey any informationregarding the actual shape of the particular elements and have beenselected for ease of recognition in the drawings.

DETAILED DESCRIPTION

The embodiments disclosed herein improve the technology of targetedadvertising by increasing the speed, accuracy and flexibility ofselection and playing of targeted television advertisements. In oneembodiment, a computerized system for targeted ad insertion, receives aprogram content stream and detects images in the content stream. Thesystem recognizes at least one visual object within the images andmatches the recognized visual object to an advertisement that correlateswith the recognized visual object. Part or all of this process of objectrecognition and matching may be performed automatically by the system ormanually. For example, if the system recognizes a particular model ofcar throughout a number of scenes in the television program, the systemmay search a database of advertisements and match the car to aparticular advertisement that advertises that car. The system may alsoor instead determine a theme, topic, category of interest and/orrelevant viewer demographic determined likely to be interested in such aprogram based on the recognition and analysis of relevant objects and/oraction sequences appearing in the images of the program content. Thesystem may then select an advertisement that advertises, correlates to,is directed to or is otherwise relevant to products or services fallingunder the theme, topic, category of interest and/or relevant viewerdemographic. The system then inserts an identifier of the matchedadvertisement at an insertion point into the program content stream.This enables playback of the matched advertisement during playback ofthe program content stream at a time that correlates to the insertionpoint into the program content stream of the identifier of the matchedadvertisement. For example, such advertisements may be automaticallyinserted during playback of the program content by the viewer's device,such as a set-top box, accessing the advertisement content, eitherremotely or locally, using the identifier of the matched advertisement.Since the advertisement matches with a particular object in the programcontent, it is relevant to the program content and likely more relevantand more accurately targeted to the viewer watching that program. Forexample, this may be due to the viewer deciding to watch that programwith such objects appearing in it generally indicating a greaterinterest of that viewer in those objects than viewers that did notdecide to watch that program.

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various disclosedembodiments. However, one skilled in the relevant art will recognizethat embodiments may be practiced without one or more of these specificdetails, or with other methods, components, materials, etc. Well-knownstructures and methods associated with media content delivery have notbeen shown or described in detail to avoid unnecessarily obscuringdescriptions of the preferred embodiments.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, suchas, “comprises” and “comprising” are to be construed in an open,inclusive sense, for example “including, but not limited to.”

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. Thus, the appearances of the phrases “in one embodiment” or“in an embodiment” in various places throughout this specification arenot necessarily all referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics may be combined inany suitable manner in one or more embodiments.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. The term “or” is generally employed in itssense including “and/or” unless the content clearly dictates otherwise.

The headings and Abstract of the Disclosure provided herein are forconvenience only and do not interpret the scope or meaning of theembodiments.

FIG. 1A is a block diagram illustrating an overview of a system 100 fortargeted advertisement insertion into a program content stream,according to one embodiment.

It is to be appreciated that FIG. 1 illustrates one example of a systemfor targeted advertisement insertion into the program content stream andthat the various embodiments discussed herein are not limited to theexemplary network of FIG. 1 . The system 100 may include a variety ofcommunication systems and may use a variety of communication mediaincluding, but not limited to, satellite wireless media, cable media orany media over which communication of broadcast television programming,on-demand programming, and/or streaming media may occur, and/or overwhich Internet communication may occur.

The system 100 includes a content server 110, which receives programcontent 120 and which may access or receive advertisement content 130.Program content 120 may be provided from one or more content providers(not shown), which provide content such as video content and/or audiocontent (hereinafter “program content”) to a distributor, such as adistributor operating the content server 110. The program content 120may include content provided from content providers such as televisionstations which provide local or national video programming, specialcontent providers which provide premium based programming orpay-per-view programming, and/or radio stations which provide audioprogramming. In addition, or in the alternative, the program content 120may include content provided from on-demand content services, streamingmedia services, or the like which provide audio, video and other typesof program content to the content server 110 for delivery of programmingto a subscriber.

Advertisement content 130 may be provided to, or accessible by, thecontent server 110 from one or more advertisement sources, which may beor include one or more databases storing advertisement content 130.Advertisement content 130 may be provided to the content server 110directly from advertisers, or from third party advertisementaggregators, services or the like.

The content server 110 receives the program content 120 and accesses orreceives advertisement content 130 over a communication network 106, andthe content server 110 is coupled to a receiving device 140 over acommunication network 108. The receiving device 140 (e.g., a set-topbox, digital video recorder (DVR), media player, smartphone or mobilecomputing device) receives the program content 120 and receives oraccesses the advertisement content 130 and connects to a presentationdevice (e.g., a television, monitor or other display device) 150 todisplay the program content 120 and advertisement content 130 receivedfrom the content server 110. In some embodiments, the receiving device140 receives or accesses the advertisement content 130 directly from theanother source outside of the content server 110 via communicationnetwork 108, such as directly from advertisers, or from third partyadvertisement aggregators, services or the like.

The communication networks 106, 108 may be or include many differenttypes of communication media, now known or later developed. Non-limitingmedia examples include telephony systems, the Internet, internets, cablenetworks, fiber optic networks, microwave networks, asynchronoustransfer mode (ATM) systems, frame relay networks, packet switchednetworks, digital subscriber loop (DSL) systems, radio frequency (RF)networks and satellite systems. Further, program content 120 and/oradvertisement content 130 may be communicated from program andadvertisement content providers to the content server 110 and/orreceiving device 140 over various combinations of media. For example, atelevision broadcast station may initially communicate program content,via an RF signal or other suitable medium, which is received and thenconverted into a digital signal suitable for transmission to the contentserver 110 over a fiber optics network. As another non-limiting example,an audio content provider may communicate audio content via its ownsatellite system to the content server 110.

The received program content 120 and/or advertisement content 130 isconverted by one or more devices (not shown) as necessary at the contentserver 110 into a suitable signal that is communicated (i.e.,“uplinked”) by one or more antennae to one or more satellites (notseparately illustrated herein, although considered part of thecommunication network 108). It is to be appreciated that thecommunicated uplink signal may contain a plurality of multiplexedprograms. The uplink signal is received by the satellite and thencommunicated (i.e., “downlinked”) from the satellite in one or moredirections, for example, onto a predefined portion of the planet. It isappreciated that the format of the above-described signals are adaptedas necessary during the various stages of communication.

An antenna that is within reception range of the downlink signalcommunicated from satellite receives the above-described downlinksignal. The antenna can be located at a customer premises. Examples ofcustomer premises include a residence, a business, a car, or any othersuitable location operable to receive signals from satellite or othercommunication systems. The received signal is communicated, typicallyover a hard-wire connection, to the receiving device 140, which convertsthe received signal into a signal suitable for communication to apresentation device 150.

The receiving device 140 may receive content partially from, or entirelyfrom, another source other than the above-described antenna. Otherembodiments of the receiving device 140 may receive locally broadcast RFsignals, and/or may be coupled to communication network 108 via anysuitable medium. Non-limiting examples of media communicatively couplingthe receiving device 140 to communication network 108 include cable,fiber optic, radio frequency, streaming or Internet media.

The receiving device 140, as well as any other devices shown in FIG. 1and/or other devices or systems described herein, may include at leastone computer processor coupled to at least one non-transitory memory,and is configured to execute computer-executable instructions to carryout, enable and/or otherwise perform applicable actions described hereinof the system 100 and the different embodiments described herein.Examples of a receiving device include, but are not limited to: atelevision converter, receiver, set-top box, television receivingdevice, television receiver, television recording device, satelliteset-top box, satellite receiver, cable set-top box, cable receiver,personal computer, media player, mobile device, tablet computing device,smart phone and/or television tuner. Accordingly, the receiving device140 may be any suitable converter device or electronic equipment that isoperable to receive programming (i.e., program content 120 andadvertisement content 130). Further, the receiving device 140 may itselfinclude user interface devices, such as buttons, switches, a display,and may include the presentation device 150. Additionally, the receivingdevice 140 may include recording capability, and may be or include adigital video recorder (DVR).

Examples of a presentation device 150 include, but are not limited to: atelevision (TV), a personal computer (PC), a media player, mobiledevice, tablet computing device, smart phone or the like. Presentationdevices 150 employ a display, one or more speakers and/or other outputdevices to communicate program content 120 and/or advertisement content130 to a user. In many implementations, one or more presentation devices150 reside in or near a customer's premises and are communicativelycoupled, directly or indirectly, to the receiving device 140.

FIG. 1B is a block diagram illustrating various example components 160of a system for targeted advertisement insertion into a program contentstream, according to one embodiment.

The components 160 may comprise of and/or implemented in correspondingsoftware, hardware or a combination thereof, and be part of or incommunication with the content server 110, receiving device 140 and/orpresentation device 150 of FIG. 1 . In one embodiment, during a livebroadcast of the program content stream 162, an image detection andrecognition component 164 of the content server 110 may receive theprogram content stream 162 and detect images in the program contentstream 162. For example, the image detection and recognition component164 may, in order to process the program content stream 162, extract andpossibly make copies of digital video frames or sequences of digitalvideo frames from the program content stream 162 as it is beingbroadcast or otherwise sent. Such video frames may comprise the imagesor be converted into digital images of one or more various digital imageformats (e.g., JPEG, Exif, TIFF, GIF, BMP, raster formats, 2D or 3Dvector formats, compound formats, stereo formats, etc.) for processingby the image detection and recognition component 164 for objectrecognition. The image detection and recognition component 164 may thenrecognize one or more visual objects within the images by processing theimages (e.g., using real-time and video processing object detectionutilizing computer vision) in a manner to detect particular products,objects, activities, characteristics, language and/or words within theframes, scenes or audio of the program content steam. For example, suchobjects may include objects that are, or are related to, products orservices advertised by particular television commercials or otheradvertisements. In some embodiments, a plurality of objects may berecognized as representing a particularly relevant action sequence orscene. For example, objects may include or represent, but are notlimited to: products such as cars, electronics, food, sporting goods,household goods, household supplies, gift items, clothes, accessories,etc.; places, such as restaurants, hotels, resorts, vacationdestinations, homes for sale, etc.; people, such as actors, celebrities,politicians, groups, etc.; or services, such as providing insurance,providing cellular, cable, satellite or Internet services, providinginformation technology (IT) services, providing tax services, etc.

Such video processing object detection performed by the image detectionand recognition component 164 may include the image detection as part ofthe video processing object detection process. The image detection andrecognition component 164 may also be configured to determine thelocation of the object in the video frame or image. In some embodiments,image detection and recognition component 164 comprises (and/orexecutes) object recognition logic that identifies and determines theobject in the image and/or the location of the object in the image. Theobject recognition logic may be software that employs any suitablemethodology that identifies the object in the image and, in someembodiments, determines the location of the object in the image. As anon-limiting example, the image detection and recognition component 164may employ edge matching techniques, divide-and-conquer searchtechniques, color and/or grayscale matching techniques, gradientmatching techniques, eigenvector analysis techniques, geometric hashingtechniques, scale-invariant feature transform (SIFT) techniques, speededup robust features (SURF) techniques, and/or template matchingtechniques.

In some embodiments, the image detection and recognition component 164may use one or more reference objects to compare with objects detectedin one or more images from the program content stream 162 to determinewhat the detected object is and/or whether the object detected isrelevant to a particular advertisement. For example, the image detectionand recognition component 164 may use a reference image of a bag of aparticular brand of potato chips and compare objects detected in theprogram content stream 162 to this reference image to determine whetherone or more of the objects detected in the program content stream 162 isor represents the particular brand of potato chips.

The image detection and recognition component 164 may also detect aplurality of visual object candidates within the images. The imagedetection and recognition component 164 may then determine at least oneof the visual object candidates is a type of visual object associatedwith one or more of a plurality of advertisements. A classification ofthe detected object may then be moved from a visual object candidate toa recognized visual object based on the determination that it is a typeof visual object associated with one or more of a plurality ofadvertisements. For example, background objects in the program contentstream 162 such as sky and trees may be detected as visual objectcandidates, but then disregarded by the image detection and recognitioncomponent 164 as not being recognized as a type of visual objectassociated with one or more of a plurality of advertisements.

The asset matching component 166 of the content server 110 may thenmatch one or more of the recognized visual objects within the images toa particular advertisement that correlates with the recognized visualobject. In one embodiment, the asset matching component 166 accesses aplurality of stored advertisement data corresponding to a plurality ofdifferent advertisements. The asset matching component 166 thendetermines which advertisement of the plurality of advertisementsadvertises a product that correlates with the recognized visualobject(s) within the images based on the accessing of the plurality ofstored advertisement data. The asset matching component 166 will thenmatch the recognized object(s) within the images to the determinedadvertisement of the plurality of advertisements. For example, the assetmatching component 166 may search a database of advertisements (e.g.,stored at the content server 110, receiving device 140 and/or anadvertisement server) and match the object recognized as the bag of aparticular brand of potato chips to a particular advertisement thatadvertises that brand of potato chips. In some embodiments, theadvertisement that advertises that brand of potato chips may beassociated with or linked to a reference image of the bag of potatochips and/or a reference image of a trademark, logo, name and/or othercharacteristic used by the image detection and recognition component 164and recognizable as or within an object detected in the program contentstream 162 by the image detection and recognition component 164.

In some embodiments, the asset matching component 166 determines therecognized visual object(s) in the images is or represents one or moreof the following: a product advertised in the advertisement; an actionsequence that relates to a product or service advertised in theadvertisement; or one or more words representing a product or serviceadvertised in the advertisement. The asset matching component 166 thenrecognizing there exists a correlation between the at least one visualobject and the advertisement based on the determination.

The asset matching component 166 may also or instead determine a theme,topic, category of interest and/or relevant viewer demographicdetermined likely to be interested in such a program based on therecognition and analysis by the image detection and recognitioncomponent 164 of relevant objects, words and/or action sequencesappearing in the images of the program content stream 162. The assetmatching component 166 may then select an advertisement that advertises,correlates to, is directed to or is otherwise relevant to products orservices falling under the theme, topic, category of interest and/orrelevant viewer demographic. Such advertisements may be selectedutilizing or otherwise based on tags or other metadata associated withsuch advertisements that indicate which of such advertisements fallunder the theme, topic, category of interest and/or relevant viewerdemographic associated with the detected objects in the program contentstream 162.

Part or all of the process of object recognition and matching may beperformed manually or automatically by the system. For example, in someembodiments, the process of object recognition and matching may beperformed manually for VOD applications where the program content isstored for playout at a later time.

In response to the asset matching component 166 matching the recognizedvisual object(s) within the images to an advertisement that correlateswith the recognized visual object(s), the identification (ID) insertioncomponent 168 then inserts an identifier of the matched advertisement atan insertion point into the program content stream. As explained in moredetail below with respect to FIGS. 2A and 2B, this enables playback ofthe matched advertisement during playback of the program content streamat a time that correlates to the insertion point into the programcontent stream of the identifier of the matched advertisement. Forexample, the identifier may be a code, number link or other metadataidentifying an advertisement that enables the advertisement to be foundin a database of advertisements by searching the database using theidentifier or otherwise using the identifier. Insertion of the IDresults in an updated program content stream 170 that may becommunicated to the receiving device 140 from the content server 110with one more identifiers of advertisements inserted therein.

After initiation of playback of the updated program content stream 170by the receiving device 140, the addressable insertion engine component172 of the receiving device 140 may then recognize the identifier of thematched advertisement during playback of the updated program contentstream 120 b. In response to recognizing the identifier in the updatedprogram content stream 120 b, the addressable insertion engine component172 may then access content of the matched advertisement. For example,such content may be accessed from a one or more databases storingadvertisement content 130. Advertisement content 130 may be provided tothe receiving device 140 from the content server 110, directly fromadvertisers, or from third party advertisement aggregators, services orthe like. In some embodiments, such advertisement content may bepre-loaded and stored on the receiving device 140 for quicker access inreal-time or near real-time as the updated program content stream 170 isbeing played by the receiving device 140 and/or accessed remotely as theupdated program content stream 170 is being played by the receivingdevice 140. The addressable insertion engine component 172 then insertsthe content of the matched advertisement into the program content streamduring playback of the updated program content stream 170 by thereceiving device 140.

In one embodiment, the system 100 may also or instead detect audio inthe program content stream 162 and recognize at least one characteristicof the audio. The system 100 then matches the recognized characteristicof the audio to an advertisement that correlates with the recognizedcharacteristic of the audio and inserts an identifier of the matchedadvertisement at an insertion point into the program content stream 162to generate the updated program content stream 170 to enable playback ofthe matched advertisement during playback of the updated program contentstream 170 at a time that correlates to the insertion point into theupdated program content stream 170 of the identifier of the matchedadvertisement. In some embodiments, the system 100 may select thematching advertisement based on a combination of detected audiocharacteristic with detected visual objects. For example, a character ina television program may say the name of a product advertised by theadvertisement and the product is concurrently or later detected to beshown in the program, thus triggering insertion of the advertisementthat advertises the product.

In some embodiments, the receiving device 140 may include recordingcapability, and may be or include a digital video recorder (DVR). Assuch, a user may record a broadcast program stream and store the programfor later playout on the presentation device 150. The updated programcontent stream 170 received by the receiving device 140 during originalbroadcasting of the content, may be stored by the receiving device 140for later playout. An advertisement may be inserted into the updatedprogram content stream 170 by the addressable insertion engine component172 upon detection of the identifier of the advertisement by thereceiving device 140 during broadcast of the updated program contentstream 170 or alternatively during playback of the recorded updatedprogram content stream 170.

As explained in more detail below with respect to FIG. 3 below, theaddressable insertion engine component 172 inserts the content of thematched advertisement into the updated program content stream 170 suchthat the matched advertisement is played by the device playout component174 of the receiving device 140 during the corresponding commercialbreak in the program during playback of the updated program contentstream 170 by the receiving device 140. In particular, the deviceplayout component 174 plays content of the matched advertisement duringthe playback of the updated program content stream 170 at a time duringplayback of the updated program content stream 170 that correlates tothe insertion point into the updated program content stream 170 of theidentifier of the matched advertisement.

FIG. 2A is a signal diagram illustrating a program content stream 120 aincluding an advertisement insertion cue 202 and an avail 214 and FIG.2B is a signal diagram illustrating an updated program content stream120 b including an advertisement insertion cue 203 with an identifier ofan advertisement 205 inserted into the advertisement insertion cue 203,according to one embodiment.

The program content stream 120 a received by the content server 110 mayinclude advertisement insertion cues, such as advertisement insertioncue 202. For example, the program content stream 120 a (which may be,for example, received external television signals) may be a MPEG-4transport stream having embedded advertisement insertion cues inaccordance with the SCTE 35 standard. SCTE 35 is a broadcast standardthat pertains to digital program insertion (DPI) in broadcasts, anddescribes the in-stream insertion of cue tones in MPEG streams to signalsplicing and insertion (e.g., advertisement insertion or insertion ofother content types) opportunities. As shown in FIG. 2A, the programcontent 120 may include an advertisement insertion cue 202 (such as amodified SCTE 35 signal) that includes a presentation time stamp (PTS)field for addressable insertion, such that the advertisement insertioncue 202 indicates to the content server 110 and/or the receiving device140 when to insert the addressable content (e.g., a local advertisement)into the programming stream for playout. The program content 120 mayinclude a broadcasting stream having an avail portion 214, meaning anadvertisement time slot, disposed between network program portions 212a, 212 b. The advertisement insertion cue 202 indicates a timing, suchas a specific frame or PTS in the received program content 120 stream,for inserting an advertisement into the avail portion 214 of the programcontent stream 120 a. As shown in FIG. 2B, the advertisement insertioncue 203 may also include other metadata inserted by the ID insertioncomponent 168 into the program content stream 120 a to generate theupdated program content stream 120 b shown in FIG. 2B. Such othermetadata may include an identifier of a particular advertisement 205 tobe inserted by the addressable insertion engine component 172 into theavail portion 214 of the updated program content stream 120 b accordingto the timing indicated by the advertisement insertion cue 203. Theadvertisement insertion cue 203 corresponds to the avail portion 214 ofthe program content stream 120 b and thus enables playback of aparticular advertisement matched to an object detected in the programcontent (identified by the identifier 205) during playback of theprogram content stream 120 b at a time that correlates to the insertionpoint into the program content stream 120 b indicated by the avail 214.Thus, during a live broadcast, the content server 110 receives theprogram content stream 120 a, generates a modified advertisementinsertion cue 203 including the identifier of an advertisement 205 andprovides the updated program content stream 120 b, including themodified advertisement insertion cue 203, to the receiving device 140.

FIG. 3 is a signal diagram illustrating a program content stream 120 bincluding an advertisement inserted into the avail 214 based on theidentifier of an advertisement 205 inserted into the advertisementinsertion cue 203, according to one embodiment.

Either of the content server 110 or the receiving device 140, uponrecognition of the advertisement insertion cue 203, inserts a particularadvertisement identified by the identifier of an advertisement 205, fromthe advertisement content 130, into the avail 214 slot of the updatedprogram content stream 120 b. The device playout component 174 of thereceiving device 140 then plays content of the inserted advertisement314 during the playback of the updated program content stream 120 c at atime during playback of the updated program content stream 120 c thatcorrelates to the insertion point into the updated program contentstream 120 c of the identifier of the inserted advertisement. In variousalternative embodiments, insertion of the advertisement 314 may beperformed upon detection of the advertisement insertion cue 203 by thecontent server 110 or by the receiving device 140.

The device playout component 174 then plays content of the advertisement314 during the playback of the updated program content stream 120 c at atime during playback of the updated program content stream 120 c thatcorrelates to the insertion point into the updated program contentstream 120 c of the identifier of the advertisement 205.

In some embodiments, the receiving device 140, the content server 110,the image detection and recognition component 164, the asset matchingcomponent 166, the ID insertion component 168, the addressable insertionengine component 172 and/or the device playout component 174 include anapplication program interface (“API”) that provides programmatic accessto one or more of their respective functions. For example, such an APImay provide a programmatic interface to one or more functions of thereceiving device 140, the content server 110, the image detection andrecognition component 164, the asset matching component 166, the IDinsertion component 168, the addressable insertion engine component 172and/or the device playout component 174 that may be invoked by otherprograms, a remote control (not shown), a mobile device (not shown), acontent provider, a program distributor, advertisers, or from thirdparty advertisement aggregators or services. In this manner, the API mayfacilitate the development of third-party software, such as variousdifferent on-demand service applications, user interfaces, plug-ins,adapters (e.g., for integrating functions of the receiving device 140,the content server 110, the image detection and recognition component164, the asset matching component 166, the ID insertion component 168,the addressable insertion engine component 172 and/or the device playoutcomponent 174 into desktop applications and mobile device applications),and the like to facilitate targeted advertisement insertion into aprogram content stream by other programs, a remote control (not shown),a mobile device (not shown), a content provider, a program distributor,advertisers, or from third party advertisement aggregators or services.

In an example embodiment, components/modules of the system 100 areimplemented using standard programming techniques. For example, theimage detection and recognition component 164, the asset matchingcomponent 166, the ID insertion component 168, the addressable insertionengine component 172 and/or the device playout component 174 may beimplemented as a “native” executable running on one or more centralprocessing units (CPUs), along with one or more static or dynamiclibraries. In other embodiments, the image detection and recognitioncomponent 164, the asset matching component 166, the ID insertioncomponent 168, the addressable insertion engine component 172 and/or thedevice playout component 174 may be implemented as instructionsprocessed by a virtual machine that executes as one or more otherprograms. In general, a range of programming languages known in the artmay be employed for implementing such example embodiments, includingrepresentative implementations of various programming languageparadigms, including but not limited to, object-oriented (e.g., Java,C++, C #, Visual Basic.NET, Smalltalk, and the like), functional (e.g.,ML, Lisp, Scheme, and the like), procedural (e.g., C, Pascal, Ada,Modula, and the like), scripting (e.g., Perl, Ruby, Python, JavaScript,VBScript, and the like), or declarative (e.g., SQL, Prolog, and thelike).

In a software or firmware implementation, instructions stored in amemory configure, when executed, one or more processors of the receivingdevice and/or the content server 110 to perform the functions of theimage detection and recognition component 164, the asset matchingcomponent 166, the ID insertion component 168, the addressable insertionengine component 172 and/or the device playout component 174. In oneembodiment, instructions cause a corresponding CPU or some otherprocessor, such as an I/O controller/processor, to receive the programcontent stream, detect images in the content stream, recognize at leastone visual object within the images, match the recognized at least onevisual object within the images to an advertisement that correlates withthe recognized at least one visual object, insert an identifier of thematched advertisement at an insertion point into the program contentstream, and play the content of the matched advertisement during theplayback of the program content stream at a time during playback of theprogram content stream that correlates to the insertion point into theprogram content stream of the identifier of the matched advertisement.

The embodiments described above may also use well-known or othersynchronous or asynchronous client-server computing techniques. However,the various components may be implemented using more monolithicprogramming techniques as well, for example, as an executable running ona single CPU computer system, or alternatively decomposed using avariety of structuring techniques known in the art, including but notlimited to, multiprogramming, multithreading, client-server, orpeer-to-peer, running on one or more computer systems each having one ormore CPUs or other processors. Some embodiments may execute concurrentlyand asynchronously, and communicate using message passing techniques.Equivalent synchronous embodiments are also supported by implementationof the image detection and recognition component 164, the asset matchingcomponent 166, the ID insertion component 168, the addressable insertionengine component 172 and/or the device playout component 174. Also,other functions could be implemented and/or performed by eachcomponent/module, and in different orders, and by differentcomponents/modules, yet still achieve the functions of the imagedetection and recognition component 164, the asset matching component166, the ID insertion component 168, the addressable insertion enginecomponent 172 and/or the device playout component 174.

In addition, programming interfaces to the data stored as part of theimage detection and recognition component 164, the asset matchingcomponent 166, the ID insertion component 168, the addressable insertionengine component 172 and/or the device playout component 174, can beavailable by standard mechanisms such as through C, C++, C #, and JavaAPIs; libraries for accessing files, databases, or other datarepositories; scripting languages such as XML; or Web servers, FTPservers, NFS file servers, or other types of servers providing access tostored data. The media content storage of the program content 120 andadvertisement content 130 may be implemented as one or more databasesystems, file systems, or any other technique for storing suchinformation, or any combination of the above, including implementationsusing distributed computing techniques.

Different configurations and locations of programs and data arecontemplated for use with techniques described herein. A variety ofdistributed computing techniques are appropriate for implementing thecomponents of the illustrated embodiments in a distributed mannerincluding but not limited to TCP/IP sockets, RPC, RMI, HTTP, and WebServices (XML-RPC, JAX-RPC, SOAP, and the like). Other variations arepossible. Other functionality could also be provided by eachcomponent/module, or existing functionality could be distributed amongstthe components/modules in different ways, yet still achieve thefunctions of the image detection and recognition component 164, theasset matching component 166, the ID insertion component 168, theaddressable insertion engine component 172 and/or the device playoutcomponent 174.

Furthermore, in some embodiments, some or all of the components of theimage detection and recognition component 164, the asset matchingcomponent 166, the ID insertion component 168, the addressable insertionengine component 172 and/or the device playout component 174 may beimplemented or provided in other manners, such as at least partially infirmware and/or hardware, including, but not limited to one or moreapplication-specific integrated circuits (“ASICs”), standard integratedcircuits, controllers (e.g., by executing appropriate instructions, andincluding microcontrollers and/or embedded controllers),field-programmable gate arrays (“FPGAs”), complex programmable logicdevices (“CPLDs”), and the like. Some or all of the system componentsand/or data structures may also be stored as contents (e.g., asexecutable or other machine-readable software instructions or structureddata) on a computer-readable medium (e.g., as a hard disk; a memory; acomputer network, cellular wireless network or other data transmissionmedium; or a portable media article to be read by an appropriate driveor via an appropriate connection, such as a DVD or flash memory device)so as to enable or configure the computer-readable medium and/or one ormore associated computing systems or devices to execute or otherwiseuse, or provide the contents to perform, at least some of the describedtechniques.

FIG. 4 is a flowchart illustrating a method 400 for inserting anidentifier of the advertisement into the program content stream duringthe playback of the program content stream, according to one embodiment.At 402, the system 100 receives the program content stream.

At 404, the system 100 detects images in the content stream.

At 406, the system 100 recognizes at least one visual object within theimages.

At 408, the system 100 matches the recognized at least one visual objectwithin the images to an advertisement that correlates with therecognized at least one visual object.

At 410, the system 100 inserts an identifier of the matchedadvertisement at an insertion point into the program content stream.This enables playback of the matched advertisement during playback ofthe program content stream at a time that correlates to the insertionpoint into the program content stream of the identifier of the matchedadvertisement.

FIG. 5 is a flowchart illustrating a method 500 for inserting thecontent of the advertisement into the program content stream during theplayback of the program content stream, according to one embodiment.

At 502, the system 100 initiates playback of a program content stream;

At 504, the system recognizes an identifier of an advertisement duringplayback of the program content stream. The identifier was inserted atan insertion point into the program content stream in response to atleast one visual object recognized within images of the content streambeing matched to the advertisement which correlates with the recognizedat least one visual object.

At 506, the system 100 accesses content of the advertisement in responseto the recognizing of the identifier of the advertisement duringplayback of the program content stream.

At 508, the system 100 inserts the content of the advertisement into theprogram content stream during the playback of the program contentstream. In one embodiment, the accessing content of the advertisement inresponse to the recognizing the identifier of the advertisement duringplayback of the program content stream may include accessing content ofthe advertisement at a set-top box from a remote advertisement server.In another embodiment, the accessing content of the advertisement inresponse to the recognizing the identifier of the advertisement duringplayback of the program content stream may include accessing content ofthe advertisement by a set-top box from local storage of the set-topbox.

FIG. 6 is a flowchart illustrating a method 600 for matching a visualobject recognized within the program content stream to an advertisementaccording to one embodiment.

At 602, the system 100 accesses a plurality of stored advertisement datacorresponding to a plurality of different advertisements.

At 604, the system 100 determines which advertisement of the pluralityof advertisements advertises a product that correlates with at least onevisual object recognized within images of a program content stream basedon the accessing of the plurality of stored advertisement data.

At 606, the system 100 matches the at least one visual object recognizedwithin the images to the determined advertisement of the plurality ofadvertisements.

The above description of illustrated embodiments, including what isdescribed in the Abstract, is not intended to be exhaustive or to limitthe embodiments to the precise forms disclosed. Although specificembodiments of and examples are described herein for illustrativepurposes, various equivalent modifications can be made without departingfrom the spirit and scope of the disclosure, as will be recognized bythose skilled in the relevant art.

The various embodiments described above can be combined to providefurther embodiments. To the extent that they are not inconsistent withthe specific teachings and definitions herein, all of the U.S. patents,U.S. patent application publications, U.S. patent applications, foreignpatents, foreign patent applications and non-patent publicationsreferred to in this specification and/or listed in the Application DataSheet are incorporated herein by reference, in their entirety. Aspectsof the embodiments can be modified, if necessary, to employ systems andconcepts of the various patents, applications and publications toprovide yet further embodiments.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

1. A computer implemented method for targeted ad insertion into aprogram content stream, comprising: receiving, by a computerized systemfor targeted ad insertion, the program content stream; detecting, by thecomputerized system for targeted ad insertion, images in the contentstream; recognizing at least one visual object within the images;matching, by the computerized system for targeted ad insertion, therecognized at least one visual object within the images to anadvertisement that correlates with the recognized at least one visualobject in response to the matching the recognized at least one visualobject within the images to an advertisement that correlates with therecognized at least one visual object, modifying an existingadvertisement insertion cue in the program content stream by inserting,by the computerized system for targeted ad insertion, an identifier ofthe matched advertisement at an insertion point into the program contentstream indicated by the existing advertisement insertion cue such thatthe modified advertisement insertion cue includes the identifier, theinserting of the identifier of the matched advertisement updating theprogram content stream enabling playback of the matched advertisementduring playback of the program content stream at a time that correlatesto the insertion point into the program content stream of the identifierof the matched advertisement; initiating, by the computerized system fortargeted ad insertion, playback of the program content stream;recognizing in the program content stream, by the computerized systemfor targeted ad insertion, during playback of the program contentstream, the identifier of the matched advertisement that was previouslyinserted at the insertion point into the program content stream;accessing, by the computerized system for targeted ad insertion, contentof the matched advertisement by using the identifier of the matchedadvertisement that was previously inserted at the insertion point intothe program content stream to search a database of advertisements duringplayback of the program content stream in response to the recognizing,during playback of the program content stream, the identifier of thematched advertisement that was previously inserted at the insertionpoint into the program content stream; and inserting, by thecomputerized system for targeted ad insertion, the content of thematched advertisement into the program content stream during theplayback of the program content stream based on the identifier of thematched advertisement that was previously inserted at the insertionpoint into the program content stream.
 2. The method of claim 1, furthercomprising: playing, by the computerized system for targeted adinsertion, the content of the matched advertisement during the playbackof the program content stream at a time during playback of the programcontent stream that correlates to the insertion point into the programcontent stream of the identifier of the matched advertisement.
 3. Themethod of claim 2, wherein the initiating playback of the programcontent stream; recognizing the identifier of the matched advertisementduring playback of the program content stream; the accessing content ofthe matched advertisement in response to the recognizing the identifierof the matched advertisement during playback of the program contentstream; and the inserting the content of the matched advertisement intothe program content stream during the playback of the program contentstream are performed by a receiving device at a customer premises. 4.The method of claim 2, wherein the playing the content of the matchedadvertisement during the playback of the program content stream at atime during playback of the program content stream that correlates tothe insertion point into the program content stream of the identifier ofthe matched advertisement is performed by a receiving device at acustomer premises.
 5. The method of claim 1, wherein the receiving theprogram content stream; the detecting images in the content stream; therecognizing at least one visual object within the images; the matchingthe recognized at least one visual object within the images to anadvertisement that correlates with the recognized at least one visualobject; and the inserting, the identifier of the matched advertisementat an insertion point into the program content stream, enabling playbackof the matched advertisement during playback of the program contentstream at a time that correlates to the insertion point into the programcontent stream of the identifier of the matched advertisement areperformed by a content server outside a customer premises.
 6. (canceled)7. (canceled)
 8. The method of claim 1, wherein the recognizing the atleast one visual object within the images includes: detecting aplurality of visual object candidates within the images; determining atleast one of the visual object candidates is a type of visual objectassociated with one or more of a plurality of advertisements; andrecognizing the at least one of the visual object candidates as the atleast one visual object within the images based on the determination. 9.The method of claim 1, wherein the matching the recognized at least onevisual object within the images to an advertisement that correlates withthe recognized at least one visual object includes: accessing aplurality of stored advertisement data corresponding to a plurality ofdifferent advertisements; determining which advertisement of theplurality of advertisements advertises a product or service thatcorrelates with the recognized at least one visual object within theimages based on the accessing of the plurality of stored advertisementdata; and matching the recognized at least one visual object within theimages to the determined advertisement of the plurality ofadvertisements.
 10. The method of claim 1, further comprising:detecting, by the computerized system for targeted ad insertion, audioin the content stream; recognizing, by the computerized system fortargeted ad insertion, at least one characteristic of the audio;matching, by the computerized system for targeted ad insertion, therecognized at least one characteristic of the audio to an advertisementthat correlates with the recognized at least one characteristic of theaudio; and inserting, by the computerized system for targeted adinsertion, an identifier of the matched advertisement at an insertionpoint into the program content stream, enabling playback of the matchedadvertisement during playback of the program content stream at a timethat correlates to the insertion point into the program content streamof the identifier of the matched advertisement.
 11. A system,comprising: at least one computer processor; and at least one memorycoupled to the at least one computer processor, the at least one memoryhaving computer-executable instructions stored thereon that, whenexecuted by the at least one computer processor, cause the at least onecomputer processor to: initiate playback of a program content stream;recognize in the program content stream, an identifier of a matchedadvertisement during playback of the program content stream, theidentifier of the matched advertisement having been inserted at aninsertion point into the program content stream in response to at leastone visual object recognized within images of the content stream beingmatched to the advertisement which correlates with the recognized atleast one visual object access content of the matched advertisement byusing the identifier of the matched advertisement that was previouslyinserted at the insertion point into the program content stream tosearch a database of advertisements during playback of the programcontent stream in response to the recognizing, during playback of theprogram content stream, the identifier of the matched advertisement thatwas previously inserted at the insertion point into the program contentstream; and insert the content of the matched advertisement into theprogram content stream during the playback of the program content streambased on the identifier of the matched advertisement that was previouslyinserted at the insertion point into the program content stream.
 12. Thesystem of claim 11, wherein the computer-executable instructions, whenexecuted by the at least one computer processor, further cause the atleast one computer processor to: play the content of the advertisementduring the playback of the program content stream at a time duringplayback of the program content stream that correlates to the insertionpoint into the program content stream of the identifier of theadvertisement.
 13. The system of claim 12, wherein the accessing contentof the matched advertisement includes accessing content of the matchedadvertisement at a set-top box from a remote advertisement server. 14.The system of claim 12, wherein the accessing content of the matchedadvertisement includes accessing content of the matched advertisement bya set-top box from local storage of the set-top box.
 15. The system ofclaim 11, wherein the computer-executable instructions, when executed bythe at least one computer processor, further cause the at least onecomputer processor to: receive a plurality of stored advertisement datacorresponding to a plurality of different advertisements; search theplurality of stored advertisement data corresponding to a plurality ofdifferent advertisements for advertisement data that is identified bythe identifier of the matched advertisement during playback of theprogram content stream; and access content of the matched advertisementin response to the recognizing the identifier of the advertisementduring playback of the program content stream based on the advertisementdata that is identified by the identifier of the matched advertisementduring playback of the program content stream.
 16. A non-transitorycomputer-readable storage medium having computer-executable instructionsstored thereon that, when executed by at least one processor, cause theat least one processor to: access a plurality of stored advertisementdata corresponding to a plurality of different advertisements; determinewhich advertisement of the plurality of advertisements advertises aproduct or service that correlates with at least one visual objectrecognized within images of a program content stream based on theaccessing of the plurality of stored advertisement data; match the atleast one visual object recognized within the images to the determinedadvertisement of the plurality of advertisements; insert an identifierof the determined advertisement of the plurality of advertisements at aninsertion point into the program content stream, enabling playback ofthe determined advertisement of the plurality of advertisements duringplayback of the program content stream at a time that correlates to theinsertion point into the program content stream of the identifier of thedetermined advertisement of the plurality of advertisements initiatingplayback of the program content stream; recognizing in the programcontent stream during playback of the program content stream, theidentifier of the determined advertisement that was previously insertedat the insertion point into the program content stream; accessingcontent of the determined advertisement by using the identifier of thedetermined advertisement that was previously inserted at the insertionpoint into the program content stream to search a database ofadvertisements during playback of the program content stream in responseto the recognizing, during playback of the program content stream, theidentifier of the determined advertisement that was previously insertedat the insertion point into the program content stream; and inserting,by a computerized system for targeted ad insertion, the content of thedetermined advertisement into the program content stream during theplayback of the program content stream based on the identifier of thedetermined advertisement that was previously inserted at the insertionpoint into the program content stream.